Using Experience Classification for Training Non-Markovian Tasks

Unlike the standard Reinforcement Learning (RL) model, many real-world tasks are non-Markovian, whose rewards are predicated on state history rather than solely on the current state. Solving a non-Markovian task, frequently applied in practical applications such as autonomous driving, financial trading, and medical diagnosis, can be quite challenging. We propose a novel RL approach to achieve non-Markovian rewards expressed in temporal logic LTL$_f$ (Linear Temporal Logic over Finite Traces). To this end, an encoding of linear complexity from LTL$_f$ into MDPs (Markov Decision Processes) is introduced to take advantage of advanced RL algorithms. Then, a prioritized experience replay technique based on the automata structure (semantics equivalent to LTL$_f$ specification) is utilized to improve the training process. We empirically evaluate several benchmark problems augmented with non-Markovian tasks to demonstrate the feasibility and effectiveness of our approach

Strong Photoluminescence Enhancement of MoS2 through Defect Engineering and Oxygen Bonding

We report on a strong photoluminescence (PL) enhancement of monolayer MoS2 through defect engineering and oxygen bonding. Micro- PL and Raman images clearly reveal that the PL enhancement occurs at cracks/defects formed during high temperature vacuum annealing. The PL enhancement at crack/defect sites could be as high as thousands of times after considering the laser spot size. The main reasons of such huge PL enhancement include: (1) the oxygen chemical adsorption induced heavy p doping and the conversion from trion to exciton; (2) the suppression of non-radiative recombination of excitons at defect sites as verified by low temperature PL measurements. First principle calculations reveal a strong binding energy of ~2.395 eV for oxygen molecule adsorbed on an S vacancy of MoS2. The chemical adsorbed oxygen also provides a much more effective charge transfer (0.997 electrons per O2) compared to physical adsorbed oxygen on ideal MoS2 surface. We also demonstrate that the defect engineering and oxygen bonding could be easily realized by oxygen plasma irradiation. X-ray photoelectron spectroscopy further confirms the formation of Mo-O bonding. Our results provide a new route for modulating the optical properties of two dimensional semiconductors. The strong and stable PL from defects sites of MoS2 may have promising applications in optoelectronic devices.Comment: 23 pages, 9 figures, to appear in ACS Nan

Characterization of a thermostable β-glucosidase from Aspergillus fumigatus Z5, and its functional expression in Pichia pastoris X33

<p>Abstract</p> <p>Background</p> <p>Recently, the increased demand of energy has strongly stimulated the research on the conversion of lignocellulosic biomass into reducing sugars for the subsequent production, and β-glucosidases have been the focus because of their important roles in a variety fundamental biological processes and the synthesis of useful β-glucosides. Although the β-glucosidases of different sources have been investigated, the amount of β-glucosidases are insufficient for effective conversion of cellulose. The goal of this work was to search for new resources of β-glucosidases, which was thermostable and with high catalytic efficiency.</p> <p>Results</p> <p>In this study, a thermostable native β-glucosidase (nBgl3), which is secreted by the lignocellulose-decomposing fungus <it>Aspergillus fumigatus </it>Z5, was purified to electrophoretic homogeneity. Internal sequences of nBgl3 were obtained by LC-MS/MS, and its encoding gene, <it>bgl3</it>, was cloned based on the peptide sequences obtained from the LC-MS/MS results. <it>bgl</it>3 contains an open reading frame (ORF) of 2622 bp and encodes a protein with a predicted molecular weight of 91.47 kDa; amino acid sequence analysis of the deduced protein indicated that nBgl3 is a member of the glycoside hydrolase family 3. A recombinant β-glucosidase (rBgl3) was obtained by the functional expression of <it>bgl</it>3 in <it>Pichia pastoris </it>X33. Several biochemical properties of purified nBgl3 and rBgl3 were determined - both enzymes showed optimal activity at pH 6.0 and 60°C, and they were stable for a pH range of 4-7 and a temperature range of 50 to 70°C. Of the substrates tested, nBgl3 and rBgl3 displayed the highest activity toward 4-Nitrophenyl-β-D-glucopyranoside (pNPG), with specific activities of 103.5 ± 7.1 and 101.7 ± 5.2 U mg^-1, respectively. However, these enzymes were inactive toward carboxymethyl cellulose, lactose and xylan.</p> <p>Conclusions</p> <p>An native β-glucosidase nBgl3 was purified to electrophoretic homogeneity from the crude extract of <it>A. fumigatus </it>Z5. The gene <it>bgl</it>3 was cloned based on the internal sequences of nBgl3 obtained from the LC-MS/MS results, and the gene <it>bgl3 </it>was expressed in <it>Pichia pastoris </it>X33. The results of various biochemical properties of two enzymes including specific activity, pH stability, thermostability, and kinetic properties (Km and Vmax) indicated that they had no significant differences.</p

Evaluation of the CALIPSO Lidar-observed particulate backscattering coefficient on different spatiotemporal matchup scales

The satellite lidar-derived ocean particulate backscattering coefficient (bbp) has rarely been validated globally with in situ observations, and we need to understand how well the satellite CALIPSO lidar bbp approach performs. Whether lidar bbp performs better in terms of observation accuracy compared to passive ocean color remote sensing has yet to be evaluated for detailed validation. With the continued deployment of the BGC-Argo float array in the global open ocean in recent years, data have accumulated with a total of 42,932 particulate backscattering coefficients (bbp) from 2010 to 2017, allowing for a finer spatial and temporal scale evaluation of the performance of the CALIPSO lidar-observed bbp. We evaluated the performance of CALIPSO-retrieved bbp products using the data detected by the BGC-Argo floats at 12 spatiotemporal matchup scales and discussed the differences in product performance at various interannual, seasonal, and spatial scales. We compare lidar, float, and ocean color bbp at the same locations and times and find that lidar bbp outperforms ocean color data. We also analyzed the key conversion factor β(π)/bbp at different spatial and temporal scales and found that there was a seasonal difference in the optimal conversion factor

Taurine alleviates Streptococcus uberis-induced inflammation by activating autophagy in mammary epithelial cells

Streptococcus uberis infection can cause serious inflammation and damage to mammary epithelial cells and tissues that can be significantly alleviated by taurine. Autophagy plays an important role in regulating immunity and clearing invasive pathogens and may be regulated by taurine. However, the relationships between taurine, autophagy, and S. uberis infection remain unclear. Herein, we demonstrate that taurine augments PTEN activity and inhibits Akt/mTOR signaling, which decreases phosphorylation of ULK1 and ATG13 by mTOR and activates autophagy. Activating autophagy accelerates the degradation of intracellular S. uberis, reduces intracellular bacterial load, inhibits over-activation of the NF-κB pathway, and alleviates the inflammation and damage caused by S. uberis infection. This study increases our understanding of the mechanism through which taurine regulates autophagy and is the first to demonstrate the role of autophagy in S. uberis infected MAC-T cells. Our study also provides a theoretical basis for employing nutritional elements (taurine) to regulate innate immunity and control S. uberis infection. It also provides theoretical support for the development of prophylactic strategies for this important pathogen